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1.  Introduction 


1.1  Background 

The  US  Army  Research  Laboratory’s  Human  Research  and  Engineering  Directorate 
(ARL/HRED)  evaluates  cockpit  design  for  new  and  upgraded  Army  aircraft  during  simulations 
and  operational  testing.  These  cockpit  evaluations  are  used  to  identify  and  eliminate  human 
factors  engineering  design  problems. 

ARL/HRED  has  evaluated  several  AH-64  Apache  attack  helicopter  cockpit  design  changes.  One 
of  the  significant  design  changes  evaluated  by  ARL/HRED  is  the  integration  of  unmanned 
aircraft  system  (UAS)  control  into  the  Apache.  Information  about  the  evaluation  methodology 
and  results  can  be  found  in  Hicks  and  Durbin  (2013). 

The  AH-64  Apache  (Fig.  1)  is  a  tandem-seat  twin-engine  aerial  weapons  platfonn  built  by 
Boeing.  The  aircraft  carries  Hellfire  missiles,  2.75-inch  rockets,  and  a  30-inm  chain  gun  and 
includes  a  fire  control  radar  and  modernized  target  acquisition  designation  sight/pilot  night 
vision  sensor  (MTADS/PNVS)  integrated  sensor  system.  There  are  2  multipurpose  displays  in 
each  cockpit  and  an  MTADS  electronic  display  and  control  in  the  copilot-gunner  (CPG)  cockpit 
(Fig.  2). 


Fig.  1  AH-64  Apache  (courtesy  of  Boeing) 
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Fig.  2  Apache  Risk  and  Cost  Reduction  System  CPG  cockpit  (courtesy  of  Camber  Corporation) 

Modern  UASs  (Fig.  3)  typically  consist  of  an  aircraft  host  platform,  a  payload  (sensors), 
weapons  when  applicable,  and  a  data  communications  suite  to  allow  for  external  (teleoperated) 
control  from  an  operator  and  data/imagery  streaming  to  command  consoles.  Depending  on  the 
scenario  and  the  UAS  type,  operators  could  consist  of  infantrymen,  aircraft  pilots,  or  dedicated 
UAS  ground  control  station  (GCS)  operators.  The  purpose  of  the  UAS  is  to  allow  for  longer  and 
more  discrete  reconnaissance  loitering,  air  support,  remote  sensor  deployment,  and  target 
engagements  without  endangering  the  UAS  operators.  Teleoperation  and  interaction  with  a  UAS 
can  be  divided  into  the  following  general  levels  of  interaction  (automation)  describing  the 
operations  that  can  be  perfonned  by  the  operator: 

•  Level  1 :  Receipt  and  transmission  of  secondary  imagery  or  data 

•  Level  2:  Receipt  of  imagery  or  data  directly  from  the  UAS 

•  Level  3:  Control  of  the  UAS  payload 

•  Level  4:  Control  of  the  UAS,  except  takeoff  and  landing 

•  Level  5:  Full  function  and  control  of  the  UAS  to  include  takeoff  and  landing 
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According  to  an  article  in  Militaiy  and  Aerospace  Electronics  (Howard  2013),  UAS  spending  is 
expected  to  double  over  the  next  decade.  These  expenditures  will  be  due  to  technological 
improvements  to  the  UAS  (e.g.,  flight  characteristics  and  sensors)  and  from  integration  with 
other  assets  (e.g.,  ground  stations,  aircraft,  and  other  UASs). 

1.2  Purpose 

The  purpose  of  this  study  was  to  investigate  the  feasibility  of  controlling  multiple  UASs  from  the 
cockpit  of  an  AH-64  Apache  helicopter.  ARL/HRED  has  conducted  simulations  and  operational 
testing  to  assess  operation  of  a  single  UAS  from  the  cockpit  of  an  AH-64.  Employment  of  a 
single  UAS  by  AH-64  crews  during  unit  training  and  subsequent  theater  operations  has  validated 
the  simulation  and  testing  results.  However,  no  simulation  or  testing  has  been  conducted  that 
investigated  the  control  of  multiple  UASs  from  the  Apache  cockpit. 

The  control  of  multiple  UASs  from  the  Apache  cockpit  can  help  crews  locate  and  identify  targets 
during  reconnaissance  and  attack  missions.  For  example,  Apache  crews  could  use  one  UAS  to 
clear  their  route  to  an  engagement  area  while  using  another  UAS  to  maintain  visual  contact  on  a 
target  in  the  engagement  area.  The  primary  human  factors  concern  for  Apache  crew  control  of 
UASs  is  excessive  mental  workload  that  could  degrade  flight  and  mission  task  performance. 
Identifying  and  integrating  the  appropriate  levels  of  UAS  control  automation  is  important  to 
ensure  safe  and  effective  mission  accomplishment. 
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This  report  will  summarize  the  workload  ratings  collected  during  single  UAS  cockpit  simulations 
and  compare  the  ratings  to  a  sample  of  ground-based  and  flight  simulations  and  test  events  that 
have  evaluated  control  of  multiple  UASs  by  operators.  Additional  research  will  be  summarized 
for  decentralized  control  and  goal/task-oriented  operations  of  UASs  to  investigate  the  use  of 
these  methods  as  options  for  Apache-UAS  teaming.  The  intent  is  to  suggest  whether  or  not 
multiple-UAS  control  by  an  Apache  crew  would  be  feasible  with  respect  to  pilot  workload  and  to 
identify  potential  solutions  (e.g.,  interface  design,  operator  feedback,  and  UAS  automation 
requirements)  that  should  be  investigated  to  reduce  workload. 

1.3  Workload 

According  to  Lu  et  al.  (2013),  human  working  memory  is  severely  limited  in  its  capacity 
duration.  As  the  information  needed  for  UAS  control  increases,  the  pilot’s  mental  workload 
rapidly  increases.  There  are  several  definitions  of  pilot  mental  workload.  A  useful  definition  is 
“the  integrated  mental  and  physical  effort  required  to  satisfy  the  perceived  demands  of  a 
specified  flight  task”  (Roscoe  1985).  It  is  important  to  assess  pilot  workload  because  mission 
accomplishment  is  related  to  the  mental  and  physical  ability  of  the  crew  to  effectively  perform 
their  flight  and  mission  tasks.  If  one  or  both  pilots  experience  excessively  high  workload  while 
perfonning  flight  and  mission  tasks,  the  tasks  may  be  perfonned  ineffectively  or  abandoned.  In 
general,  workload  assessment  techniques  are  used  to  answer  the  following  questions  (Eisen  and 
Hendy  1987): 

•  Does  the  operator  have  capability  to  perform  the  required  tasks? 

•  Does  the  operator  have  enough  spare  capacity  to  take  on  additional  tasks? 

•  Does  the  operator  have  enough  spare  capacity  to  cope  with  emergency  situations? 

•  Can  the  task  or  equipment  be  altered  to  increase  the  amount  of  spare  capacity? 

•  Can  the  task  or  equipment  be  altered  to  increase/decrease  the  amount  of  mental  workload? 

•  How  does  the  workload  of  a  new  system  compare  to  the  old  system? 

To  assess  whether  the  pilots  are  task-overloaded  during  a  mission,  the  level  of  workload  for  each 
pilot  must  be  evaluated.  Spare  workload  capacity  is  an  important  commodity  for  pilots  because 
they  are  often  required  to  perfonn  several  tasks  concurrently.  For  example,  copilots  often 
perfonn  navigation  tasks,  communicate  via  multiple  radios,  monitor  aircraft  systems,  and  assist 
the  pilot  on  the  controls  with  flight  tasks  (e.g.,  maintain  airspace  surveillance)  within  the  same 
time  interval.  Mission  perfonnance  is  reduced  if  pilots  are  task-saturated  and  have  little  or  no 
spare  capacity  to  perform  other  tasks  (Durbin  and  Hicks  2009)  such  as  UAS  control. 

Currently  several  types  of  subjective  workload  evaluation  surveys  are  used,  2  examples  of  which 
are  the  Bedford  Workload  Rating  Scale  (BWRS)  and  the  National  Aeronautics  and  Space 
Administration  Task  Load  Index  (NASA-TLX). 


4 


The  BWRS  has  been  used  extensively  by  the  military,  civil,  and  commercial  aviation 
communities  for  pilot  workload  estimation  (Roscoe  and  Ellis  1990).  It  requires  pilots  to  rate  the 
level  of  workload  associated  with  a  task  based  on  the  amount  of  spare  capacity  they  feel  they 
have  to  perform  additional  tasks.  After  each  mission,  pilots  rate  flight  and  mission  tasks  using 
the  BWRS  scale  (Fig.  4).  Roscoe  and  Ellis  (1990)  describe  the  BWRS  and  explain  its  use  in 
assessing  pilot  workload.  ARL/HRED  has  written  the  requirement,  for  several  Anny  aircraft, 
that  operation  of  the  aircraft  must  not  result  in  workload  ratings  of  more  than  5. 0-6.0  on  the 
BWRS  scale.  Flight  and  mission  tasks  that  are  rated  5.0  or  higher  are  evaluated  to  detennine  if 
crewstation  design  problems  contributed  to  higher  workload  ratings. 


Workload  Description 


Is  workload 
satisfactory 
without  reduction  in 
spare  (workload)  capacity? 


NO 


Is  workload  tolerable 
for  the  task? 


NO 


Is  it  possible  to 
complete  the  task? 


Pilot  Decisions 


Rating” 


Workload  insignificant 

1 

Workload  low 

2 

Enough  spare  capacity  for  all 
desirable  additional  tasks 

3 

Task  abandoned.  Pilot  unable  to 
apply  sufficient  effort 


Insufficient  spare  capacity  for  easy 
attention  to  additional  tasks 

4 

Reduced  spare  capacity.  Additional 
tasks  cannotbe  given  the  desired 
amount  of  attention 

5 

Little  spare  capacity:  level  of  effort 
allows  little  attention  to  additional 
tasks 

6 

Very  little  spare  capacity,  but 
maintenance  of  effort  in  the  primary 
tasks  not  in  question 

7 

Very  high  workload  with  almost  no 
spare  capacity.  Difficulty  in 
maintaininglevel  of  effort 

8 

Extremely  high  workload.  No  spare 
capacity.  Serious  doubts  as  to  ability 
to  maintain  level  of  effort 

9 

10 


Fig.  4  Bedford  workload  rating  scale 
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The  NASA-TLX  (Hart  2006)  consists  of  6  subscales  that  represent  somewhat  independent 
clusters  of  variables:  mental,  physical,  and  temporal  demands  plus  frustration,  effort,  and 
perfonnance  (Fig.  5).  The  assumption  is  that  some  combinations  of  these  dimensions  are  likely 
to  represent  the  “workload”  experienced  by  most  people  performing  tasks.  A  weighting  scheme 
is  used  to  derive  the  overall  workload  score  based  on  the  6  subscales.  More  information  about 
the  development  of  the  NASA-TLX  can  be  found  in  Hart  and  Staveland  (1988).  NASA-TLX  has 
been  used  extensively  in  documenting  user-based  task-oriented  workload. 

NASA  Task  Load  Index 

Hart  and  Staveland's  NASA  Task  Load  Index  (TLX)  method  assesses 
work  load  on  five  7 -point  scales.  Increments  of  high,  medium  and  low 
estimates  for  each  point  result  in  21  gradations  on  the  scales. 


Name 

Task 

Date 

Mental  Demand 

1  1  1  1  1  1  1 

How  mentally  demanding  was  the  task? 

1  II  1  1  1  1  1  1  1  1  1  1  1 

Very  Low 

Physical  Demand 

1  1  1  1  1  1  1 

Very  High 

How  physically  demanding  was  the  task? 

1  1  1  1  1  1  1  1  1  1  1  1  1  1 

Very  Low 

Temporal  Demand 

1  1  1  1  1  I  I 

Very  High 

How  hurried  or  rushed  was  the  pace  of  the  task? 

1  1  1  1  1  1  1  1  1  1  1  1  1  1 

Very  Low 

Performance 

Very  High 

How  successful  were  you  in  accomplishing  what 

1  1  1  1  1  1  1 

you  were  asked  to  do? 

1  1  1  1  1  1  1 

1  1  1  1  1  1  1 

Perfect 

Effort 

Failure 

How  hard  did  you  have  to  work  to  accomplish 

1  1  1  1  1  1  1 

your  level  of  performance? 

1  1  1  1  1  1  1 

1  1  1  1  1  1  1 

Very  Low 

Frustration 

Very  High 

How  insecure,  discouraged,  irritated,  stressed. 

1  II  1  1  1  1 

and  annoyed  wereyou? 

1  1  1  1  1  1  1 

1  1  1  1  1  1  1 

Very  Low 

Very  High 

Fig.  5  NASA-TLX  assessment 
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1.4  Level  of  Automation 

Level  of  automation  (LOA)  refers  to  the  level  of  task  planning  and  performance  interaction 
maintained  between  a  human  operator  and  computer  (robot)  in  controlling  a  complex  system 
(Billings  1991,  Kaber  and  Endsley  2004).  Identifying  the  LOA  necessary  for  successful  human- 
robot  interaction  is  vital  for  overall  safety,  efficiency,  and  mission  success.  If  the  LOA  is  too 
low,  operator  workload  can  become  excessive.  If  the  LOA  is  high,  it  is  important  to  ensure  that 
the  UAS  is  initiating  and  conducting  proper  actions;  otherwise,  the  operator  could  lose  trust  in 
the  system  and  choose  not  to  employ  it.  A  balance  must  be  formed  where  the  UAS-operator 
interaction  does  not  result  in  excessive  operator  workload  but  still  provides  enough  confidence 
that  ordered  tasks  could  be  successfully  completed.  There  has  been  significant  work  done  in  the 
development  of  LOA  taxonomy,  although  different  applications  employ  different  taxonomies  for 
describing  the  automation.  A  proposed  LOA  taxonomy  (Endsley  and  Kaber  1999,  Kaber  and 
Endsley  2004)  shows  the  relationship  between  the  operator  interface  and  the  responsibilities  of 
the  human  and  computer  as  follows: 

•  Manual:  Human  performs  all  tasks. 

•  Action  Support:  System  assists  the  operator  with  performance  of  a  selected  action. 

•  Batch  Processing:  Human  selects  options  to  be  performed  and  the  system  carries  the 
actions  out  automatically. 

•  Shared  Control:  Human  and  computer  generate  possible  decision  options,  human  selects 
options,  and  actions  are  shared  between  human  and  the  system. 

•  Decision  Support:  Computer  generates  a  list  of  decision  options  that  the  human  can  select 
from.  Once  the  option  is  selected,  the  computer  performs  the  action. 

•  Blended  Decision  Making:  Computer  generates  a  list  of  decision  options,  which  it  selects 
and  carries  out  if  the  human  consents.  The  human  can  modify  the  options  if  necessary. 

•  Rigid  System:  Presents  only  a  limited  set  of  options  to  the  operator.  The  operator  must 
select  from  these  options;  no  modifications  are  allowed. 

•  Automated  Decision  Making:  System  selects  the  best  option  to  implement  and  carries  out 
that  action,  based  on  a  list  of  alternatives  it  generates  (or  the  human  operator). 

•  Supervisory  Control:  System  generates  options,  selects  the  option  to  implement,  and  carries 
out  the  action.  The  human  monitors  the  system  and  intervenes  if  necessary. 

•  Full  Automation:  System  carries  out  all  actions;  the  human  is  out  of  the  loop  and  cannot 
intervene. 

These  operator-system  LOAs  can  be  dynamic  and  change  depending  on  the  type  of  system  and 
operational  scenario.  In  this  case,  the  term  adaptive  automation  (AA)  is  used  to  describe  the 
variation  of  LOAs  to  meet  operator  workload  requirements  and  assist  in  the  decision-making 
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process.  The  study  conducted  by  Kaber  and  Endsley  (2004)  describes  AA  as  the  process  by 
which  control  must  pass  back  and  forth  between  the  human  and  the  automation  over  time, 
depending  on  situational  demands,  and  seeks  to  find  ways  of  exploiting  this  process  to  increase 
human  performance  and  reduce  high  levels  of  operator  workload. 

The  studies  presented  in  the  following  sections  examine  research  conducted  on  the  operator-UAS 
interface  at  varying  LOAs  ranging  from  full  manual  control  to  full  automation. 


2.  Literature  Review  of  Control  Methods  and  Findings 


2.1  Overview 

The  following  subsections  summarize  data  collected  from  literature  reviews  of  select  UAS 
control  reports  and  their  results.  Experimental  methodology  is  generalized  and  overall  results 
provided  for  each  identified  test  and  experiment.  The  data  will  be  used  to  develop  conclusions 
and  recommendations  in  the  follow-on  section  regarding  control  of  multiple  UASs  by  Apache 
helicopter  crews. 

2.2  ARL/HRED  Simulation  and  Operational  Testing 

Previous  workload  data  collected  by  ARL/HRED  related  to  single  UAS-cockpit  operation  has 
been  documented  in  simulation  and  operational  test  reports.  Workload  data  collected  for 
nonintegrated  Apache -UAS  teaming  was  published  in  a  report  by  Hicks  et  al.  (2009)  entitled 
AH-64D  Apache  Longbow/Video  from  UAS  for  Interoperability  Teaming  Level  II  ( VUIT-2 ) 
Aircrew  Workload  Assessment.  During  this  simulation,  the  pilots  conducted  realistic  mission 
scenarios  while  teaming  with  the  UAS  and  an  associated  GCS  operator.  Level  2  UAS  (imagery 
data  only)  was  provided  to  the  CPG.  As  missions  were  conducted,  the  CPG  would  instruct  the 
GCS  operator  where  to  position  the  UAS  and  what  terrain,  personnel,  and  cultural  features  to 
identify  with  the  UAS  sensor.  BWRS  data  were  collected  at  the  end  of  each  mission;  the  average 
workload  rating  for  VUIT-2  missions  for  CPGs  was  3.3  and  3.0  for  non- VUIT-2  missions.  The 
pilot-on- the-controls  (PI)  workload  ratings  for  the  VUIT-2  missions  were  2.6  and  for  non-VUIT- 
2  missions  were  3.7.  These  workload  ratings  showed  that  it  was  feasible  to  incorporate  UAS 
level  2  imagery  data  into  the  Apache  cockpit  during  representative  mission  scenarios  without 
causing  excessive  crew  workload. 

A  simulation  was  conducted  on  an  integrated  system  that  allowed  the  Apache  crew  to  interact 
with  the  UAS  up  to  control  level  4  (full  control,  except  takeoff  and  landing).  The  simulation 
methodology  and  results  were  published  in  a  report,  AH-64D  Apache  Longbow  Aircrew 
Workload  Assessment  for  Unmanned  Aerial  System  (UAS)  Employment ,  by  Durbin  and  Hicks 
(2009).  As  missions  were  conducted,  the  CPG  would  operate  the  UAS  sensor  (level  3),  control 
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the  UAS  flight  path  (level  4),  or  instruct  the  GCS  operator  where  to  position  the  UAS  and  what 
features  to  identify  with  the  UAS  sensor  (level  2).  BWRS  data  were  collected  at  the  end  of  each 
mission  and  the  average  workload  rating  for  UAS  missions  for  CPGs  was  2.9  while  that  for 
non-UAS  missions  was  3.0.  PI  average  workload  for  UAS  missions  was  2.6  and  that  for 
non-UAS  missions  was  2.8.  These  workload  ratings  showed  that  it  was  feasible  to  incorporate 
UAS  control  levels  2-4  into  the  Apache  cockpit  during  representative  mission  scenarios  without 
causing  excessive  crew  workload.  In  general,  pilots  commented  that  UAS  control  workload  is 
slightly  higher  than  “normal”  missions  but  manageable,  and  the  situational  awareness  benefits 
gained  by  external  sensors  outweighed  the  slight  increase  in  workload. 

Findings  by  ARL/HRED  showed  that  integrating  UAS  levels  2-4  into  the  Apache  helicopter  did 
not  significantly  increase  crew  workload  or  negatively  impact  crew  performance  and  mission 
accomplishment.  During  the  evaluations,  subject  matter  experts  (SMEs)  independently  rated 
whether  the  missions  were  a  success  or  failure.  The  criteria  that  the  SMEs  used  to  rate  mission 
success  or  failure  was  whether  the  aircrew  completed  most  or  all  of  their  mission  objectives  and 
did  not  crash  or  get  shot  down.  In  both  evaluations,  pilots  completed  100%  of  missions 
successfully  when  operating  a  UAS.  These  independent  ratings  support  the  conclusion  that  crews 
can  control  one  UAS  during  missions  without  experiencing  excessive  workload. 

ARL/HRED  has  helped  conduct  operational  testing  for  the  AH-64.  During  operational  testing, 
the  pilots  maintained  UAS  control  levels  2,  3,  and  4  at  various  times  during  missions.  The 
workload  ratings  and  mission  success  ratings  collected  by  ARL/HRED  further  support  the 
conclusion  that  crews  can  operate  one  UAS  without  experiencing  excessive  workload. 

2.3  Human-UAS  Interaction  Literature  Review  and  Past  Studies 

A  study  by  Ruff  et  al.  (2002),  Human  Interaction  with  Levels  of  Automation  and  Decision  Aid 
Fidelity  in  the  Supervisory  Control  of  Multiple  Simulated  Unmanned  Air  Vehicles,  investigated  3 
LOAs — manual  control,  management  by  consent,  and  management  by  exception — to  control 
UASs  in  a  dynamic  simulation  environment.  This  study  was  used  to  identify  techniques  that  may 
allow  multiple-UAS  control  by  a  ground  operator.  The  levels  of  automation  were  defined  as  the 
following: 

•  Manual  Control:  Automation  is  donnant  unless  initiated  by  the  operator. 

•  Management  by  Consent:  Automation  proposes  action  but  cannot  act  without  explicit 
operator  consent. 

•  Management  by  Exception:  Automation  acts  without  explicit  operator  consent  and  fails  to 
act  only  when  commanded  by  the  operator. 

This  study  used  performance  measures  of  efficiency,  correct  decision  responses,  and  event 
management  along  with  subjective  NASA-TLX  workload  measurements  to  evaluate  the  different 
LOAs.  The  results  of  this  study  showed  that  manual  control  for  a  single  UAS  resulted  in  the 
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lowest  workload,  and  the  highest  workload  was  when  2-4  UASs  were  supervised  during  the 
scenario.  Management  by  consent  resulted  in  a  lower  workload  than  management  by  exception 
for  1  and  2  UASs  but  higher  for  4  UASs.  This  article  also  examined  “trust”  of  the  UAS.  It  was 
found  that  if  the  UAS  acted  in  a  manner  that  seemed  incorrect  or  outside  of  the  expected  actions, 
trust  in  the  automation  was  lower.  The  study  concluded  that  management  by  consent  helped 
humans  remain  in  the  loop  of  control  without  requiring  constant  vigilance  of  status  indicators. 
Increasing  the  automation  level  reduced  the  situational  awareness  of  the  operator  and  generally 
lowered  operator  trust  if  some  form  of  inaccuracy  occurred.  The  primary  suggestion  from  this 
study  was  to  examine  different  levels  of  (variable)  automation  depending  on  mission  scenarios 
and  tasks  to  assist  the  operators  in  accomplishing  mission  objectives. 

An  article  by  Cummings  and  Morales  (2005),  UA  Vs  as  Tactical  Wingmen:  Control  Methods  and 
Pilots  ’  Perceptions,  examined  the  use  of  the  previously  discussed  control  methods  (manual, 
control  by  consent,  and  control  by  exception)  and  surveyed  US  Air  Force  pilots  to  obtain 
feedback.  A  simulation  was  set  up  to  provide  the  pilots  an  interface  that  depicted  the  UAS  as  a 
wingman,  and  pilots  performed  a  cognitive  walkthrough  of  the  interface  to  identify  potential 
usability  problems  and  characterize  the  problem-solving  process.  The  results  of  the  study 
indicated  that  Air  Force  (fixed-wing)  pilots  generally  preferred  management  by  consent  as  the 
control  strategy  that  would  allow  them  to  avoid  performing  tedious  functions  and  still  allow  them 
the  final  decision  with  respect  to  UAS  actions  during  missions.  The  pilots  did  not  want  the  UAS 
to  make  decisions  about  target  engagement  or  to  perfonn  any  tasks  that  might  interfere  with  the 
manned  aircraft  airspace  requirements.  Additionally,  the  article  suggested  that  the  complex 
socio-technical  cultural  component  of  single-seat  versus  multicrew  cockpits  could  have  a 
significant  effect  on  the  acceptance  of  UAS  technology  in  the  cockpit. 

A  study  conducted  by  Fern  and  Shively  (2009),  A  Comparison  of  Varying  Levels  of  Automation 
on  the  Supervisory  Control  of  Multiple  UASs,  indicated  that  new  concepts  of  operations  and 
procedures  were  required  for  a  single  (ground)  operator  to  simultaneously  control  multiple 
UASs.  This  study  investigated  the  use  of  an  interface  technique  called  Playbook,  which  allowed 
operators  to  designate  specific  “plays”  that  were  tied  to  semi-autonomous  UAS  tasks  in  an  effort 
to  reduce  operator  workload.  A  play  example  would  be  for  the  operator  to  call  for  “Overwatch 
Tango”,  which  would  provide  the  UAS  instructions  to  loiter  over  the  Tango  waypoint.  This 
experiment  focused  on  3  levels  of  automation:  Playbook  (multiple  UAS),  script  (single  UAS), 
and  tools  (no  automation).  Performance  measurements  collected  included  accuracy,  speed,  and 
target  identifications.  The  results  from  this  study  showed  that  the  Playbook  interface  resulted  in 
better  perfonnance  than  the  other  methods,  and  the  NASA-TLX  data  indicated  that  when  using 
the  Playbook,  workload  was  significantly  lower  than  using  scripts  or  tools.  The  study  showed  the 
advantages  of  task  delegation  to  support  supervisory  control  of  multiple  UASs. 
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Studies  conducted  by  Lu  et  al.  (2013)  and  Hou  and  Kobierski  (2006)  investigated  human- 
computer  interface  designs  that  could  reduce  workload  when  controlling  UASs.  Lu  et  al.’s 
Design  and  Test  of  a  Situation-Augmented  Display  for  an  Unmanned  Aerial  Vehicle  Monitoring 
Task  (2013)  focused  on  using  an  augmented  display  that  would  provide  operators  with  important 
situational-dependent  UAS  status  information.  The  results  of  this  study  showed  that  the 
situation-augmented  display  improved  operator  response  times  to  abnormalities  and  reduced  total 
flight  time  required  to  complete  objectives.  However,  at  the  conclusion  of  the  study  it  was  found 
that  while  perfonnance  was  better  for  a  situational-augmentation  display,  the  reported  workload 
was  not  significantly  different  from  a  nonaugmented  display.  The  study  further  reported  that 
UAS  monitoring  tasks  are  difficult  and  that  providing  high-level  situation  awareness  infonnation 
through  visual  displays  may  improve  overall  operator  perfonnance. 

Hou  and  Kobierski’s  2006  study,  Intelligent  Adaptive  Interfaces:  Summary  Report  on  Design, 
Ddevelopment,  and  Evaluation  of  Intelligent  Adaptive  Interfaces  for  the  Control  of  Multiple 
UAVs  from  an  Airborne  Platform,  investigated  the  development  and  design  of  intelligent 
adaptive  interfaces  (IAIs)  for  the  control  of  multiple  UASs.  Their  findings  showed  that  when 
using  IAI  automation  agents  and  aids,  UAS  operators  performing  cognitively  complex  tasks  with 
high  workload  were  able  to  reduce  the  time  required  to  perform  critical  tasks.  Six  system 
functional  groups  were  identified  for  IAI  (situational-based  decision-making)  investigation  and 
proposed  experimentation:  inter-crew  communications,  route  planning,  route  following,  screen 
management,  data-link  monitoring,  and  UAS  sensor  selection.  The  results  of  the  study  showed 
that  using  an  IAI  to  monitor  and  perfonn  decision-based  predefined  UAS  operations  resulted  in 
reduced  operator  workload  and  increased  situational  awareness. 

A  study  conducted  by  Calhoun  et  al.  (2004),  Tactile  Versus  Aural  Redundant  Alert  Cues  for  UA  V 
Control  Applications,  considered  the  use  of  tactile  and  aural  alert  cues  for  UAS  control 
applications.  The  study  found  that  aural  and  tactile  cues  can  reduce  the  visual  workload  required 
for  UAS  task  monitoring,  but  no  significant  differences  for  response  time  to  alerts  between  the 
aural  and  tactile  cues  were  observed  during  the  study. 

2.4  UAS  Autonomous  Decision-making  Literature  Review  and  Past  Studies 

Maintaining  crew  workload  at  an  acceptable  level  is  necessary  to  employ  multiple  UASs  from 
the  cockpit.  A  variety  of  methods  have  been  used  to  reduce  the  burden  placed  on  operators  by 
providing  more  automation  capabilities  to  operators  when  interacting  with  a  UAS  network.  The 
following  studies  examined  multi-agent  systems,  task  allocation  architectures,  and  decentralized 
control  frameworks  for  autonomous  UAS  control  with  minimal  operator  input. 

A  study  conducted  by  Lian  and  Deshmukh  (2006),  Performance  Prediction  of  an  Unmanned 
Airborne  Vehicle  Multi-agent  System,  examined  the  use  of  a  Markov  Decision  Process  to 
determine  optimal  paths  for  UAS  mission  execution  when  considering  the  use  of  UASs  as 
control  agents  in  a  dynamic  multi-agent  system.  The  premise  of  the  study  was  to  develop  a 
decision-making  process  that  allowed  the  UASs  to  share  data  with  each  other  to  accomplish  a  set 


11 


of  mission  goals  (e.g.,  avoid  gunfire,  successfully  reach  targets,  negotiate  paths  within  system 
constraints).  The  study  found  that  when  using  the  decentralized  model,  and  when  UASs  share 
data  and  group  dynamically,  heuristic  algorithms  allowed  the  UASs  to  effectively  identify 
optimal  directions  (paths)  to  targets. 

A  paper  by  Lemaire  et  al.  (2004),  A  Distributed  Tasks  Allocation  Scheme  in  Multi-UAV  Context, 
proposed  a  completely  distributed  architecture  where  robots  dynamically  allocate  their  tasks 
while  building  their  own  plans.  The  paper  investigated  incremental  task  algorithms  and 
parameters  that  would  distribute  and  balance  the  workload  equally  among  different  robots. 
Dynamic  and  temporary  hierarchies  were  developed  for  task  auctions  when  time  constraints 
between  tasks  were  present.  The  study  focused  on  token-based  auctions  to  distribute  tasks 
amongst  the  UASs.  When  robots  were  idle  or  their  hierarchy  chain  allowed,  they  could  bid  on 
tasks  (i.e.,  go  to)  created  by  other  robots  to  help  accomplish  the  overall  mission  requirements.  An 
auction  leading  robot  (one  that  has  either  been  assigned  a  token  or  generated  an  initial  token) 
would  allow  other  robots  to  bid  on  the  tasks  that  were  available  to  the  auction  leader.  Once  all  of 
the  tasks  were  bid  and  distributed,  another  robot  could  request  the  token  and  begin  auctioning 
their  tasks  to  reduce  single  robot  workload.  The  resulting  simulation  showed  that  simple 
planning  algorithms  using  the  token-based  task  distribution  system  could  be  used  to  accomplish 
required  tasks  among  multi-robot  systems. 

A  study  by  Cheng  et  al.  (2013),  Cooperative  Control  ofUAV  Swarm  via  Information  Measures, 
investigated  a  rule-based  decentralized  control  framework  for  a  swann  of  UASs  carrying  out  a 
cooperative  ground  target  engagement  mission  scenario.  In  this  case  the  behaviors  of  the  UASs 
were  governed  by  rule  sets  that  lead  to  system-level  cooperation.  Infonnation  measures  were 
adopted  to  estimate  the  value  of  future  actions,  and  a  prediction  model  was  used  to  enhance  team 
perfonnance  when  coupled  task  constraints  were  present.  An  example  scenario  from  the 
simulation  used  data  fusion  where  2  UASs  track  a  target  and  provide  information  back  to  an 
attacking  UAS  to  help  integrate  and  determine  flight  trajectories  and  firing  constraints.  The  study 
found  that  highly  cooperative  group  perfonnance  could  be  achieved  without  the  use  of  a 
centralized  controller.  However,  the  performance  was  related  to  the  complexity  of  the  coupled 
constraints  and  the  accuracy  of  the  prediction  model.  In  that  case  an  increased  number  of  UASs 
could  reduce  task  coupling.  Also,  predictive  modeling  can  help  to  allow  agents  to  estimate  the 
intentions  of  other  agents  (UASs)  and  allow  the  original  agents  to  choose  goal  oriented  activities 
that  enhance  overall  team  utility. 
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3.  Conclusions 


3.1  Summary  of  Findings 

The  data  presented  in  this  report  provides  an  overarching  look  into  the  different  test 
methodologies  and  potential  system  designs  that  could  be  used  to  facilitate  the  implementation  of 
multiple  UAS  control  from  an  Apache  cockpit.  Two  primary  concerns  that  are  present 
throughout  the  literature  are  the  appropriate  LOA  for  system  control  and  cognitive  workload 
concerns  for  the  operator.  While  a  direct  comparison  of  ARL/HRED  UAS  workload  data  and 
other  workload  study  data  cannot  be  accomplished  because  of  different  collection  and  test 
methodologies,  trends  can  be  identified  among  all  of  the  studies.  Generally,  as  fully  manual 
operation  of  UASs  decreases,  operator  workload  decreases  as  well,  which  is  to  be  expected  when 
automatic  control  is  present  in  the  system  and  the  automation  is  increasingly  taking  over  operator 
tasks.  However,  in  some  instances,  operators  reported  that  workload  can  increase  if  the 
infonnation  presented  for  supervisory  control  acknowledgements  is  not  presented  in  an 
appropriate  manner.  Operators  can  become  overloaded  with  supervisory  control  tasks  while 
trying  to  perform  other  tasks. 

Studies  that  dealt  with  the  human-robot  interface  and  relationship  found  that  most  operators 
preferred  to  have  some  control  over  the  situation.  Whether  the  control  was  management  by 
supervision  or  the  Playbook  interaction,  operators  needed  to  feel  that  they  had  control  over  the 
actions  of  the  UAS  to  trust  the  system.  Other  studies  examined  the  adaptive  and  intuitive 
interface  designs  to  reduce  workload  and  enhance  situational  dependent  decision-making  for  the 
operators.  In  all  cases,  operator  workload  was  reduced  by  using  a  decision-making  tool  or 
interface  to  assist  in  multiple  UAS  control. 

The  studies  that  examined  the  interaction  of  multiple  autonomous  UASs  found  that  multi-agent 
systems,  data  fusion,  and  goal-oriented  decision-making  made  it  possible  for  multiple  UAS 
systems  to  perform  tasks  collectively.  The  development  of  data-sharing  and  task-allocation 
algorithms  allows  the  UASs  to  self-detennine  flight  task  plans,  avoid  interference,  and  trade  off 
tasks  amongst  each  other.  In  these  cases,  the  UASs  operated  mostly  independently  and 
accomplished  tasks  without  significant  user  inputs  past  the  beginning  of  the  mission. 

3.2  Discussion 

An  examination  of  the  studies  presented  provides  insights  into  future  development  of  multi-UAS 
control  from  a  multi-UAS  systems  level  and  an  operator-UAS  interface  domain.  It  is  clear  from 
the  research  that  operating  multiple  UASs  can  induce  high  cognitive  workload  requirements  for 
operators  when  some  form  of  interface  assistance  or  autonomous  programming  is  not  used. 

Tasks  that  require  significant  operator  vigilance  increase  the  cognitive  demands  required  during 
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missions.  For  UAS  ground  control  operators,  high  workload  can  reduce  perfonnance  during 
missions.  For  crews  (i.e.,  pilots  and  copilots),  high  workload  can  reduce  their  perfonnance  and 
increase  the  probability  of  a  crash  or  engagement  by  the  threat. 

A  combination  of  automation  processes  and  levels  of  automation  must  be  used  to  fully  integrate 
multi-UAS  control  into  an  Apache  helicopter.  Single  UAS-Apache  control  with  cooperation 
from  a  GCS  has  been  proven  to  be  effective,  but  a  significant  increase  in  workload  would  likely 
occur  when  adding  additional  UASs.  A  multi-agent  approach  that  increases  the  amount  of 
automation  and  data  fusion  used  between  UASs  would  require  the  least  amount  of  human 
vigilance  and  task  monitoring  during  operations.  However,  the  system  would  need  to  be 
demonstrated  extensively  to  ensure  that  a  level  of  trust  between  the  UASs  and  pilots  is 
established.  As  the  control  algorithms  and  UAS  technology  improve,  having  a  UAS  wingman 
and  controlling  multiple  UASs  during  mission  scenarios  appears  to  be  viable  from  a  workload 
perspective.  However,  in  the  immediate  future,  detennining  the  correct  amount  of  automation 
from  a  supervisory  perspective  will  be  a  key  factor  in  providing  multi-UAS  capability  to  the 
Apache  as  soon  as  possible. 

Using  the  conclusions  drawn  from  the  literature  review,  a  combination  of  supervisory  control 
interfaces  (i.e.,  Playbook)  and  adaptive  interfaces  to  present  key  UAS  and  mission  parameters 
depending  on  the  situation  or  workload  of  the  pilot  would  be  beneficial  when  providing  multiple 
UAS  control  capability  to  Apache  crews.  Additionally,  an  investigation  into  integration  of  tactile 
and  aural  cues  should  be  conducted  to  assist  in  UAS-pilot  operation.  Past  studies  have  shown 
that  tactile  cueing  and  aural  alerts  can  supplement  visual  systems  by  alerting  the  pilots  to 
information  being  presented.  The  use  of  these  augmentations  must  also  be  implemented  with 
caution  to  ensure  that  only  pertinent  information  is  presented  so  as  not  to  overload  the  pilot  with 
unnecessary  cues.  Pilots  would  need  adequate  training  on  the  capabilities  and  limitations  of  the 
various  LOAs  with  respect  to  UAS  control  and  the  appropriate  amount  of  automation  required 
for  different  mission  scenarios.  Pilot  workload  should  be  assessed  to  identify  whether  the  pilot  is 
capable  of  making  automation  decisions  for  multi-UAS  control  in  high-workload  environments. 
In  a  case  where  workload  becomes  overwhelming,  the  UASs  should  employ  adaptive  automation 
to  ensure  safe  operations  throughout  a  mission  scenario. 

The  dynamic  flight  and  mission  environment  is  a  significant  challenge  when  designing  for  the 
UAS-pilot  interaction.  Multi-UAS  control  could  occur  in  cooperation  with  a  GCS  operator.  In 
these  instances  a  multi-UAS  system  could  be  split  to  share  command  authority  between  the  GCS 
operator  and  pilot.  During  this  scenario  the  pilot  would  not  have  to  maintain  awareness  of  all 
UASs;  rather,  just  the  ones  that  were  related  to  the  pilot  tasks.  To  reduce  the  workload  in 
dynamic  environments,  multi-UAS  employment  should  be  limited  to  goal-oriented  automation, 
where  the  UAS  teams  accomplish  well-defined  goals  automatically  and  independent  of  the  pilot 
(i.e.,  monitor  waypoint,  track  target,  update  plan)  and  the  pilot  has  the  ability  to  monitor  video 
feeds  or  imagery  results  from  those  actions.  In  these  instances,  expected  workload  would  be 
similar  to  single  UAS  tasks.  Using  the  UAS  teams  as  extended  sensors  (controlling  only  the 
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payloads)  is  another  option.  A  GCS  operator  could  place  the  UAS  teams  in  appropriate  locations, 
and  the  pilots  could  take  control  over  the  payload  for  reconnaissance  and  targeting  tasks.  Once 
the  pilot  was  done,  he  could  hand  the  task  back  to  the  GCS  operator  or  provide  a  command  to  the 
UAS  to  carry  out  a  predefined  action. 


4.  Proposed  Solutions,  Evaluation  Plan,  and  Future  Integration 


4,1  Proposed  Solutions 

The  use  of  adaptive  automation  and  supervisory  controls  plays  a  central  role  in  the  sample  of 
UAS  management  methodologies  discussed  in  this  report.  Using  the  Playbook  style  of  goal- 
oriented  task  direction  and  adaptive  automation  for  high-workload  environments  should  allow 
Apache  helicopter  pilots  to  control  multiple  UASs  during  missions.  In  addition,  using  different 
sensory  inputs  (i.e.,  visual,  auditory,  tactile),  as  described  by  Wickens’s  (1984)  Multiple 
Resource  Model,  could  reduce  workload  by  not  overloading  any  single  sensory  channel. 

In  the  near  tenn,  multiple-UAS  control  by  Apache  pilots  could  be  accomplished  by  allowing 
pilots  to  alternate  UAS  sensor  video  feeds  inside  the  cockpit  in  a  very  controlled  method.  This 
would  allow  multiple  battlefield  sensors  to  be  accessed  through  quickly  changeable  video  feeds 
on  the  pilot  displays.  Conceptually  the  workload  should  be  somewhat  similar  to  the  operation  of 
a  single  UAS  sensor  from  the  cockpit.  The  additional  task  of  swapping  UAS  sensors  and  gaining 
situational  awareness  has  potential  to  raise  pilot  workload  but  could  easily  be  evaluated  in  a 
simulation  environment.  Most  of  the  solutions  presented  in  this  report  deal  with  long-term 
automation  solutions,  but  the  first  step  is  to  evaluate  low  level  automation  control  of 
manipulating  UAS  sensors  and  receiving  imagery  data.  Once  the  initial  evaluations  are 
completed,  advancing  the  automation  levels  and  overall  fidelity  of  system  control  can  be 
conducted  to  improve  the  UAS  control  by  pilots. 

It  is  proposed  that  experimentation  should  be  conducted  to  use  goal-oriented  decision-making  in 
conjunction  with  adaptive  automation  and  visual,  auditory,  and  tactile  cueing  to  examine  the 
appropriate  levels  of  automation  required  for  control  of  multiple  UASs  by  Apache  helicopter 
crews.  The  use  of  multiple  decision  aids  and  cues  should  help  to  reduce  pilot  workload  by 
providing  information  to  multiple  human  input  channels,  rather  than  only  using  the  visual 
system.  The  combination  of  these  techniques  should  provide  an  effective  system  for  controlling 
multiple  UASs  from  the  cockpit  of  an  Apache.  Also,  as  UAS  designs  and  software  become  more 
advanced,  continued  testing  should  be  conducted  to  examine  the  UAS-pilot  interactions.  As  UAS 
intelligence  and  autonomy  increases,  more  mission  tasks  could  be  carried  out  by  the  UAS  with 
less  input  from  pilots.  The  optimal  design  solution  should  be  detennined  though  simulation  and 
operational  testing.  An  ideal  solution  would  likely  consist  of  highly  autonomous  UASs  with 
some  goal-oriented  inputs  from  pilots. 
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4.2  Evaluation  Plan 

To  assess  the  feasibility  of  multiple  UAS  control  by  Apache  pilots  based  on  the  sample  of 
research  summarized  in  this  report,  an  evaluation  approach  is  proposed.  The  first  step  in  the 
evaluation  is  to  interview  SMEs  on  the  proposed  control  tasks  that  Apache  pilots  would 
potentially  use  during  representative  missions  to  ensure  that  the  levels  of  automation  proposed 
for  further  research  are  reasonable  and  feasible  for  mission  types.  SMEs  can  provide  valuable 
input  on  the  use  and  employment  of  the  aircraft  and  UAS  use.  Once  the  mission  is  understood, 
simulations  can  be  conducted  using  modeling  software  (e.g.,  Improved  Perfonnance  Research 
Integration  Tool,  or  IMPRINT)  to  analyze  mission  tasks  and  workload  interactions  between  the 
aircraft-UAS-mission  goals.  These  simulations  can  be  adjusted  for  sensitivity  and  the  number  of 
UASs  controlled  can  be  adjusted  to  represent  high-level  task  perfonnance.  A  more  detailed  look 
at  multi-UAS  control  requires  integration  of  the  proposed  solutions  into  a  manned  simulator  to 
evaluate  pilots  interacting  with  various  design  solutions  and  receiving  pilot  feedback  and 
objective  data  collection  (i.e.,  task  time  and  button  presses).  The  final  evaluation  of  the  proposed 
solutions  would  be  to  take  the  solutions  that  perform  best  in  simulation  and  integrate  them  into 
the  actual  aircraft  for  full  operational  testing  and  evaluation.  As  data  are  collected  during 
operational  testing,  incremental  improvements  can  be  added  and  retested  to  arrive  at  a  preferred 
solution. 

4.3  Future  Integration 

The  future  integration  of  multi-UAS  control  into  an  Apache  cockpit  must  be  investigated 
carefully.  Controlling  multiple  UASs  can  provide  a  tactical  advantage  to  pilots  by  increasing 
situational  awareness  and  reducing  the  probability  of  detection  and  engagement  on  the 
battlefield.  Operating  multiple  UASs  from  an  Apache  has  potential  to  benefit  the  crew  if  the 
following  are  achieved: 

•  The  appropriate  levels  of  automation  are  identified. 

•  The  automation  supports  high  levels  of  trust. 

•  Proper  controls  are  implemented  for  adaptive  automation  in  high-workload  environments. 

•  Consideration  is  given  to  multi-agent  systems  and  data  fusion. 

•  The  pilot-UAS  control  interface  is  developed  using  accepted  human  factors  design 
standards  and  practices. 

•  The  automation  is  focused  on  minimizing  pilot  workload. 

•  Appropriate  tactics,  techniques,  and  procedures  are  developed  for  aiding  the  crew  when 
controlling  multiple  UASs. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AA 

adaptive  automation 

ARL 

US  Army  Research  Laboratory 

BWRS 

Bedford  workload  rating  scale 

CPG 

copilot/gunner 

GCS 

ground  control  station 

HRED 

Human  Research  and  Engineering  Directorate 

IAI 

intelligent  adaptive  interfaces 

LOA 

level  of  automation 

MTADS 

modernized  target  acquisition  designation  sight 

NASA-TLX 

National  Aeronautics  and  Space  Administration  Task  Load  Index 

PI 

pilot  on  the  controls 

PNVS 

pilot  night  vision  sensor 

SME 

subject  matter  expert 

UAS 

unmanned  aircraft  system 

UAV 

unmanned  aerial  vehicle 

VUIT-2 

video  from  UAS  for  interoperability  teaming  level  II 
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